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As an initial attempt to document the influence of 
class size reduction on overall student achievement, this report 
evaluates Project Challenge, a Tennessee Department of Education 
program to reduce at-risk student/ teacher ratio in kindergarten 
through third grade classes in 17 rural Tennessee school systems. A 
summary of findings relating to the Student Teacher Achievement Ratio 
Project (Project STAR) and the Lasting Benefits Study is followed by 
discussion of the design and limitations of Proiect Challenge. The 
Tennessee Comprehensive Assessment Program (TCAP) achievement test 
was used to evaluate student achievement, and Project Challenge 
students were compared with students from Tennessee's 138 school 
systems. Results showed that from 1990 to 1991, 9 of the 17 systems 
in Project Challenge improved their statewide rankings in reading, 
and 10 Project Challenge systems improved their statewide rankings in 
mathematics. Continuous study is recommended as a means of 
documenting definitive effects from reduced class size and related 
school practices on student achievement* Eight tables of data and two 
figures are included, and a sample classroom information survey is 
appended. (MH) 
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Abstract 



The Tennessee Department of Education's innovative approach for using 
Federal Chapter I funds, and over four million dollars in state funds, to reduce 
kindergarten through third-grade class sizes in "poor" rural schools has resulted 
in the opportunity for 17 Tennessee school systems to offer more effective 
classroom practices. The Department's 1985-89 Student Teacher Achievement 
Ratio (STAR) Project concluded that reduced class-sizes in grades K-3 produced 
statistically and educationally significant achievement benefits. The pervasive 
and positive results in sti^dent gains of this large-scale, longitudinal experiment 
which utilized random assignment of students, and the continuation of these 
gains as shown in the Lasting Benefits Study (LBS) warranted the implementation 
of class-size reduction for schools in the lowest per capita (and highest percent of 
free/reduced lunch participation) income counties in Tennessee. Project 
Challenge, the program to reduce class sizes for schools in 17 counties, has 
produced these initial results: 1) an average increase of 5.3 ranks in reading and 
2) and an average increase of 6.6 ranks in mathematics in the ranking of 
Tennessee's 138 school systems on the Tennessee Comprehensive Assessment 
Program (TCAP). This amounts to nearly a quarter (.25) of a standard deviation 
gain. These initial positive findings merit continuous study of improved 
achievement test scores. A period of analysis of at least three years is dkuirable 
to document definitive effects from reduced class-size and related school 
practices designed to improve student achievement. Overall the Project 
Challenge schools' systems scores for students in second grade are moving 
positively toward the State mean. Analysis using refined data will allow the 
benefits of this intervention to be analyzed and documented more thoroughly. 
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PROJECT CHALLENGE: AN INITTAI. EVALUATION OF 
TENNESSEE'S RURAL "AT-RISK" STUDENT/TEACHER RATIO 
REDUCTION PROJECT IN SEVENTEEN COUNTIES 

Project Challenge 

Based on the pervasive and positive Project STAR findings, the Tennessee 
Department of Education initiated a project to implement system-wide reduction uf 
elementary grades K-3 class size within school districts located in Tennessee's 
poorest counties. Ultimately 17 local school systems met the high percentage 
criteria for free/reduced-lunch participation for selection to participate in Project 
Challenge, a class-size reduction effort to improve academic achievement for "at- 
risk" students in Tennessee, The systems were located in the lowest per-capita 
income counties in the State. Most (15 out of the 17) operate in rural Appalachian 
communities in East Tennessee; two are in West Tennessee (See Figure 1). 

Federal Chapter I fiinds supplemented the costs for small classes within 
individual schools that met the federal requirements. Beginning in the 1989-90 
school year, to permit system-wide implementation, the Department allocated 
State funds to the remaining schools in selected school systt?ms that did not meet 
the federal Chapter I guidelines, but which had 60 to 74 percent of students who 
qualified for free or reduced lunch. 

Through Project Challenge, Tennessee's Department of Education has 
provided an opportunity for Tennessee school systems located in the poorest 
counties to offer their students and teachers the advantages (e.g., more personal 
interaction, individualized instruction, close monitoring, immediate feedback^ 
etcO of small classes. This evaluation is an initial attempt to document the 
influence of class-size reduction on overall student achievement, without 
controlling for other school interventions, assuring random student assignment, 
and without engaging in any special testing or providing staff training* 
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The Tennessee Department of Education requested tliat The Center of 
Excellence for Research in Basic Skills at Tennessee State University begin to 
evaluate the effectiveness of Project Challenge in relation to the Lasting Benefits 
Study (LBS) being conducted by the Center, The Center began in 1990 
conducting the longitudinal, follow-up study to determine the lasting benefits of 
small classes for Project STAR students, and is now beginning to evaluate the 
subsequent implementation of reduced class-size in the 17 participating Project 
Challenge school systems in terms of student achievement. The research is 
jointly supported by a Tennessee Department of Education contract and the 
Center of Excellence for Research in Basic Skills. 

Background 

Summary of Findings for Project STAR and LBS. The Student Teacher 
Achievement Ratio Project (Project STAR), Tennessee's four-year longitudinal 
experimental class-size study, demonstrated that reduced class size (1:15) in 
kindergarten through third grade (K-3) significantly enhanced student 
achievement. The Project STAR study insured random assignment of students to 
three class-size interventions: small classes (13 to 17 students per teacher), 
regular classes (22 to 25), and regular classes with a fiill-time teacher's aide. 
The sample contained over 6,000 students in 42 districts and as many as 79 
schools classified as inner-city, rural, urban, or suburban. Overall findings of 
Project STAR indicated a significant (statistically and educationally) achievement 

advantage (specifically in reading and mathematics) for students who were in 

small classes. The longitudinal follow-up of Project STAR, the Lasting Benefits 
Study (LBS), has shown that the achievement benefits for students in grades K-3 
small classes are retained at least two years after (through fifth grade) the student 
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leaves the smail-class condition. Even after rwo years , fifth-grade students who 
had been in grades K-S small classes maintain a statistically significant 
achievement advantage o^^er the fifth-graders who had been in Project STAR 
regular classes and classes with a fiill-time aide. 

Students in all locations (rural, urban, suburban, and inner-city) benefited 
from small classes in grades K-3. Small-class students in rural areas scored 
highest on academic achievement tests in general, although inner-city students 
(who were performing below the State mean) in grades K-3 scored the greatest 
gains. In the LBS analysis the small-class treatment in grades K-3 benefited all 
students regardless of school system geographic location. 

The Project STAR research also supports the existing class-size literature 
regarding the use of more desirable teaching practices in small classes. 
Johnston's (1990) analysis of a large number of teacher interviews found that 
grades K-3 small-class teachers reported engaging in teaching practices that, in 
accordance with Bredekamp (1987), are considered developmentally appropriate 
and congruent with the early childhood education literature. 

Project STAR results confirm that teachers of small classes are also able to 
address children's individual social and emotional needs and problems (Word et 
al., 1990). Given the changing nature of families and of the increasing poverty- 
related needs of children (Hamburg, 1992; Hodgkinson, 1991), this attention may 
be important in redefining the teaching processes in early primary grades. Small- 
class teachers indicated that they had better knowledge of children as individuals, 
their families and their home backgrounds. These teachers reported that their 
relations with children were improved; and that children's relations with each 
other were more positive. The extent to which teachers and children were 
friendly, supportive, and trusting of one another was an indication of the peer 
cohesion of children and the "esprit de corps" of the group as a whole (Johnston 

' 8 



ERIC 



BEST COPY AVAILABLE 



& Davis, 1989). This dimension of positive interaction is an indicator of 
classroom morale and team spirit that is characteristic of effective elementary 
schools. 

Project STAR is currently regarded as the definitive class-size study. The 
Project STAR study has been cited as ''the most significant educational research 
done in the U.S. during the past 25 years" (Orlich, 1991, p. 632). Project STAR 
results have prompted administrators in some states to reduce elementary class 
sizes, and at the very least the findings have contributed to a national reevaluatioa 
of the class-size issue by educational policy makers at all levels (Gillman, 1990; 
Nye et ah, 1992; Folger, 1989/92). 

Findings from Project STAR and other class-size studies show that small 
classes can be both emotionally and academically beneficial to young children. If 
teachers in Project Challenge schools can focus on each child's psychological, 
social, and academic needs through smaller classes, this could provide all 
students and particularly low-income, educationally at-risk children with the extra 
boost needed to help them become academically successful and perhaps 
contribute to their overall well-being. 

Class-size reduction can be expensive and, therefore, may be considered as 
less feasible to implement than other interventions. However, other interventions 
have not been shown to be as uniformly successful as class size reduction in 
terms of raising overall student achievement. Also some interventions have 
hidden costs such as long-term specialized staff training, resource personnel 
costs, and other required support services (Nye, et aL, 1992). 

New federal Chapter I flexibility has allowed state and local cooperative 
efforts for reducing class size to become a reality in several Tennessee school 
systems. The Hawkins-Stafford Amendment of 1988 has made it possible for 
schools serving a large number of children from low-income families to use 



federal Chapter I fuMs foK all students, not just for those labeled "low 
achievers." Federal Chapter I funding is available to fund state projects that 
improve the total educational program of a school. State educators are permitted 
to design projects that fit their specific educational objectives. Projects are only 
open to schools with student enrollment of 75% or greater who are from families 
with poverty incomes, as measured by federal free-lunch program participation. 
Within these guidelines, state leaders are permitted to select schools for project 
participation. Tennessee has taken this opportunity to enhance student 
achievement through reducing class size for schools that meet this criteria. 

Design and Limitations of the Project Challenge Evaluation 

Historical test data comparison. Prior to the 1989-90 school year, Tennessee 
schools were required to administer the Stanford Achievement Test (SAT) for 
students in 2nd, 5th, and 7th grades. The Basic Skills First (BSF) test was 
administered to students in the 3rd, 6th and 8th grades. Beginning in 1989-90 
students in the 2nd through 8th grades and the 10th grade were required to take 
the Tennessee Comprehensive Assessment Program (TCAP) achievement test. 
The TCAP includes a norm-referenced test (NRT) and a criterion-referenced test 
(CRT) component* 

Since no special testing was prescribed for Project Challenge, the extant 
data and present TCAP testing processes had to be used in the evaluation plan. 
Furthermore, because there are no means by which to compare SAT (1988-89) 
with TCAP (1989-90) scores directly, the 1989-90 TCAP second-grade results 
had to be used as the "baseline" data for Project Challenge. This baseline 
selection is preferred to the alternative use of pre-Project Challenge (no time in 
small classes) SAT results because of the confidence achieved from the 
comparison of TCAP scores from year to year. Consequently, the 1989-90 

Id 

BEST COPY AVAILABLE 

ERIC 



second-grade student "baseline" sample already had one year of small-class 
treatment, and their scores are compared to the 1990-91 TCAP test results of 
second-grade students who have had two years in small classes, (See Table 1.) 



Tabie 1 

Project Challenge (1990-93): Summery of Student Participation & 
Years Tested by the TN Comprehensive Assessment Program 

(TCAP) 



Testing Year Grade-2 students' small-class experience in 

(Date) rrCAP) Project Challenge (number of years) by grade(s) 

Test Date Years of Grades in 

Project Challenfl e Prolect Challenoe 

1990 1 grade two only 

1991 2 grades one & two 

1992 3 grades Ki one & two 

1993 4 grades K, one & two 



Ultimately, it was necessary to extrapolate some analysis information from 
Tennessee's Project STAR (Word et al,, 1990) in order to design aspects of the 
Project Challenge evaluation. Project STAR researchers found that the 
achievement gains for a second-grade sample with no prior small-c^.ass 
experience were not so large as to suggest a dramatic positive change in their 
scores at that grade level. The treatment occurs at the time the student enters the 
small-class condition for the first time, so the 1989-90 second-grade "baseline" is 
contaminated, but it seems to be the most appropriate data available for this 
purpose. 

Student roster analysis. General demographic data were collected through the 
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Classroom Information Survey (Appendix A). This survey indicated that for the 
1990-91 school year most class sizes ranged from 13 to 17 students per teacher. 
(See Figure 2.) However 25% of the kindergarten, 12% of the first-grade, 10% 
of the second-grade, and 13% of the third-grade classes exceeded 17 students 
(this is the maximum number of students to be considered a small class. Word et 
al,). Should exact data concerning the average class sizes in grades K-3- one 
year prior to the initiation of Project Challenge(1988) through the 1990 school 
year-become available, it may be possible to be more precise in some of the 
analyses and summary statements. This will involve collecting class rosters 
during 1992*93 of the ongoing study if future funding permits Project Challenge 
data analysis. 

Intervening VariabJes. Since Project Challenge is not an "experiment" there 
was no random selection or assignment of students and no special testing, etc.. 
The evaluation essentially employs an after-the-fact (post hoc) review and 
analysis of grouped (second-grade, system-wide) data. There were no controls 
for other special "interventions" that may have occurred during the Project 
Challenge implementation. Therefore, .^valuation staff have no means by which 
to make conclusions regarding achievem^jnt gain (or loss) solely as a result of the 
class-size reduction. There may also be other systematic threats to the validity of 
the preliminary findings, such as the presence of inflated class sizes when schools 
could not fiilly implement class reductions. 

These initial evaluation results will help establish baselines and trends for the 
class-size reduction effort in Project Challenge systems. Data from these systems 
will need to be analyzed more thoroughly to determine student achievement 
gains. Accurate interpretation will require tracking through student rosters to 
ensure class-size reductions over time and data on other school system-wide 
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interventions in the Project Challenge counties that may also influence pupil 
progress. 

Grouped data . Grouped data by grade level are inherently susceptible to 
variations in student ability by classes or by grades. Gains (or losses) in one year 
may be the result of very good (or very poor) student ability, excellent teaching, 
test variation, etc. It is only with several continuous years of results tha* a 
definite trend can become evident. 

Preliminary Evaluation Findii^s 

Results based on rankings of systems. One method to ascertain if Project 
Challenge class-size reduction (1 :15) appears to be making some impact on 
academic achievement was the comparison of the 17 systems' ranking among the 
138Tennessee systems based upon second-grade (1989-90 and 1990-91) TCAP 
scaled scores. This was done for both reading and mathematics by adding the 
rankingji, dividing (by 17) and comparing the resulting averag e ranks on 
mathematics and reading. (See Table 2.) Of the 17 Project Challenge systems, 
from 1990-1991, nine improved their rankings in reading and ten improved in 
mathematics, This is a gross measure since it does not take into account the 
amount of the movement from their 1988-89 to 1989-90 rank. Generally, the 
Project Challenge systems as a whole are moving closer to the State average. As 
shown in Table 2, Challenge systems gained an average of 5.3 ranks in reading 
and 6.6 ranks in math. The average rank of the Project Challenge systems (94 in 
reading and 79 in math) is still considerably below the state average of 69 (of 138 
systems). 
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Table 2 



Rankings of Project Challenge Districts (n = 17) of 138 Tennessee 
School Systems (1990 and 1991) Based on Grade-Two Reading and 
Mathematics Scores Measured by the TN Comprehensive 
Assessment Program (TCAP)* 



Reading 
1 989-90 1990-91 



Mathematics 
1999-90 1990-91 



Sum of 
Ranl<s 



16''j1 



1591 



1448 



1336 



Divided by 
<n = 17) 



98.9 



93.6 



85.2 



78.6 



Difference in 
1990 and 1991 



Gain (-1-901 
in total of ranks 



Gain (-1-112) 
In total of ranks 



Divided by 
(n = 17) 



Average Gain 
5.3 ranks 



Average Gain 
6.6 ranks 



'^Uslng 138 systems as the base, the average district rank is 69. 



Results based on Z-score comparisonsL. A second procedure entailed the 
conversion of the school system's average TCAP scores to Z-scores and then the 
determination of extent to which the 17 Project Challenge systems' second-grade 
average scores in reading and mathematics deviate (i.e., in terms of standard 
deviation units) from the state average. Again, using the 1990 and the 1991 
TCAP testing results, some gains are evident. (See Table 3.) 




Table 3 



Comparison of Project Challenge Systems (n = 17) Average Z- 
Scores for Reading and Mathematics, 1989-90 to 1990-91, 
Grade Two, TN CQmprehensiye Assessment Prograjn Results 



Reading Mathematics 
Year 1 989-90 1990-91 1 989-90 1990-91 

2-Score -.75 -.52 -.34 -.08 

Difference Gain of .23 Gain of .26 

Although the average Z-scores for both reading and mathematics and for both 

1990 and 1991 TCAP test results for the 17 systems are below the state average, 
there has been nearly a quarter (.25) of a standard deviation gain. Thus, these 17 
systems, between 1990 and 1991, have moved closer to the State TCAP 
mathematics and reading mean test scores. 

Vie gains in rankings and in Zrscore comparisons suggest that, on average, 
the second-grade results on TCAP indicated achievement benefits that might be 
attributed to participation in small classes: student scores are getting better as 
the systems move closer to the median state ranking. Tables 4 through 8 provide 
more details regarding Z-score comparisons and rank-order differences. 

Table 4 shows that nine of the Project Challenge systBms improved their 
state ranking on the TCAP Reading sub-score between the end of the 1990 and 

1991 school years. The range of increase in rankings was from 1 to 117 ranks. 
Eight Project Challenge systems had a decrease in rank ranging anywhere from 1 
to 62 rank positions. The mean increase in rank was 5 positions. 
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Table 5 indicates that ten of the Project Challenge school systems improved 
their rank position on the TCAP Math sub-score between the end of the 1990 and 
1991 school years. The range of increase in rankings was from 1 to 99 rank 
positions. Seven Project Challenge school systems had a decrease in rank 
ranging from 4 to 75 rank positions. Overall, the mean difference in rank was 
increased 7 positions. 

Table 6 denotes twelve Project Challenge school systems moving upward in 
the rankings on the TCAP Total Language sub-score . The range of increase in 
rankings was from I to 86 ranks. Five Project Challenge school systems had a 
decrease in rank ranging from 2 to 80 positions. Overall, the mean difference in 
rank was increased 4 positions. 

Table 7 shows that eleven i:f the Project Challenge school syslems improved 
Jflifiir rank position on the TCAP Word Analysis sub-score between the end of the 
1990 and 1991 school years. The range of increase m rankings was from 1 to 45 
ranks. Five Project Challenge school systems decreased in rank ranging from 11 
to 43 positions. Overall, the mean difference in rank was increased 8 positions. 

Table 8 denotes eight Project Challenge systems moving upward in the 
rankings on the TCAP Science sub-score . The range of increase in rankings was 
from 2 to 1 15 ranks. Nine Project Challenge school systems had a decrease in 
rank ranging from 3 to 38 rank positions. Overall, the mean difference in rank 
was increased 8 positions. 

lajsummary , the Project Challenge schools have moved up (on average) in 
their rankings of Tennessee school systems on the second-grade TCAP test. This 
trend should be monitored and analyzed in more detail to see if it is logical to 
attribute a substantial portion of this gain to the small-class (1: 15) intervention. 
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Future Analyses 

To obtain more precise information on student achtevement changes » 
evaluators need exact counts of the numbers of students/teacher in all target 
grades for each year of the Project Challenge implementation and for at least one 
year prior (1988-89). This is particularly important as the analysis (second-grade 
averages) presumes that die changes occurred in all grades rather than the major 
gains occurring at the time of treatment, This analysis may greatly understate 
any positive gains from small classes* 

Another way to check on gains In Urget grades (K-3) will be through 
analyses of Chapter I data from as many Prulect Challenge school systems as 
possible. Given the poverty levels In Ihese systems, it Is? probable that most, if 
not all students in the Project Challenge sohools will b@ eligible for Chapter 1. If 
this is the case, it would be useful to identify Chapter I schools and classes that 
consistently implement 1:15 class^-alzet ratios and analyxQ only these classes, This 
analysis might be initiated after the othsr data^CQlliOtlon possibilities are 
examined and if the other options for more refined data cotleotlon do not seem 
feasible. This option would not require additional testings but would require 
coordination between the Tennessee Department of Educalion and the evaluators 
in collecting the required data. 

Summary and Recommendations Coiic^rniiig Ciass-Sls&e Reduction in 
Project Challenge Counties 

Nationally, rural areas have at ieasl 30% of the population living below the 
poverty level (Helge> 1988), Thus, rural schools are often characterized by a 
large proportion of economically disadvantaged students. The declining need for 
farm workers and the closing of local mines and small industries in rural areas 

2'J 



(kji 



have created high unemployment rates > often above the national average. A 
myriad of family problems accompany economic problems of high unemployment 
and poverty and often leave children victims of broken homes or intense family 
stress ("End of the Road," 1988), 

Other conditions associated with rural environments include health issues such 
as high teen pregnancy and low birth-weight baby rates (Hodgkinson, 1985), 
limited opportunity for various careers due to geographic isolation, limited 
availability of entertainment and cultural enrichment, reduced access to support 
services due to traasportation problems, and lower family support for education 
("End of the Road," 1988) due to lower literacy levels in rural areas. These and 
other factors place a significant number of students "at-risk" of educational 
difficulty or failure in school. 

In addition to high unemployment or under employment at minimum wage or 
part-time work, rural communities are plagued with low property assessment 
values, a condition that directly impacts the amount of funds available for 
education (Honeyman et al., 1989). Many rural schools operate without adequate 
facilities, support services, and educational tools and materials (e.g,, computers). 
For example, many schools do not have guidance counselors to address the "at- 
risk" students' developmental and psychological needs. 

Small classes implemented in the early primary grades may mitigate some 
circumstances with which rural at-risk children are forced to cope and rural 
children in general experience- Bredekamp (1987) asserts that the developmental 
appropriateness of an early childhood education program depends most on the 
direct interactions between adults and children. Thus, a significant reduction of 
class size should make a positive contribution to developmen tally appropriate 
practice in primary-grade classrooms because of increased possibilities for 
student- teacher interactions. The class-size-reduction intervention in rural 
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schools may be particularly cost-effective since some classes are already smaller 
as a result of the geographic location in less populated rural communities. 

Further, small-size classes foster more developmentally appropriate non- 
academic interpersonal interactions among the children and between teachers and 
children. Robinson and Wittebols (1986) conclude that smaller classes tend to 
promote the use of more desirable teaching practices such as one-on-one 
interactions with each child, and "individualization" of instruction. These "non- 
academic" improvements contribute to the child's overall sense of self-esteem, 
which has been shown to enhance the child's ability to master "academic" 
knowledge and skills » (Ramey, 1992), 

The preliminary evaluation of Project Challenge findings and the research 
literature support the continuing reduction qf class size In the Project Challenge 
counties as one important strategy to improve student academic scores, Finally, 
the Lasting Benefits Study is adding to the knowledge base that ciass-size 
reduction in the primary grades provides an environment to increase students' 
level of participation in school activities (Finn et al., 1992), and that this type of 
early intervention will have long-term social and economic benefits (e.g., 
Weikart, 1989; Zigler, 1992). 
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APPENDIX A 

LASTING BENEFITS STUDY 
Project Challenge 
Classroom Information Survey 



Please provide the following information about yourself and 
your current class. Please disregard the numbers in parentheses 
to tha right of the page. 



Label Here **** 



TEACHER IKPORMATION 



Name: 



Social Security No. : 

Sex: (circle one) 
Race: (circle one) 

Grade (circle one) 
Level 



1 
1 

K 
2 



Male 2 
White 2 



Female 
Black 3 



Kindergarten 1 
2nd Grade 3 



= Other 

1st Grade 
3rd Grade 



CLASS IMF0IU4ATI0N 

1. No. of Students on class roll, May 7 
a« No. of White Students 

b. No, of Black Students 

c. Ho. of Other Race Students 

d» No. of Students on Free or Reduced Lunch 

e. No. of Students Recommended for 
Promotion to Next Grade 

2. No. of Students in Attendance, May 7 



3. I provide primary instruction in the following 



subject areas: 

a. Language Arts (not reading) 

b. Math 

c. Reading 

d. Science 

e. Social Science 

f . Other Academic Subjects 



(Please Circle) 

1 « Yes 2 No 

1 = Yes 2 - No 

1 = Yes 2 = No 

1 « Yes 2 No 

1 = Yes 2 = No 

1 = Yes 2 = No 



Por 

Data Entry 
Use only 

(22-31) 



(32-40) 



(41) 
(42) 

(43) 



(45-46) 
(47-48) 
(49-50) 
(51-52) 
(53-54) 
(55-56) 

(57-58) 



(60) 
(61) 
(62) 
(63) 
(64) 
(65) 
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